1. Technical Field
This application relates to managing analysis of activity data.
2. Description of Related Art
Not long ago, people communicated important information between one another through the physical delivery of paper. Delivering documents in this way to convey important information once dominated business but has since been largely displaced by electronic delivery and communication. Whether it is by email or otherwise, today people send many sensitive and important documents and information electronically. The movement to electronic distribution of information has increased businesses' awareness of security issues. Electronic files are easy to copy and transmit out of an unwitting organization. Potential saboteurs like hackers, for example, can access, steal, alter, and/or destroy important information.
This increased awareness in security issues concerning electronic communications led companies to begin to monitor data transfers between entities, such as people, computers, and resources. The enormous volume of data generated by communications between entities (e.g., people viewing websites, people sending emails to one another, people transferring files to one another, and many other communications) made it difficult for a company to monitor all of the communication information. To help alleviate this problem, companies developed systems that analyze communications to determine which communications are likely illegal or otherwise prohibited by the companies' business rules.
In addition, because of the many ways to communicate over a network and the many different analysis tools needed to perform network forensics, the conventional method makes it difficult to answer even simple questions such as “What is happening on my network?,” “Who is talking to whom?,” and “What resources are being accessed?” It is difficult because there is no limit as to which applications one can use. Each application introduced onto a network brings new protocols and new analytical tools to audit those applications. For example, there are many ways to send a file to another person using a network: E-mailing the document as an attachment using the SMTP protocol; transmitting the file using an Instant Messenger; uploading the file to a shared file server using the FTP protocol; web sharing the document using the HTTP protocol; or uploading the file directly using an intranet protocol like SMB or CIFS. All of these protocols are implemented differently and special analysis tools are required to interpret them; a complex and expensive system.
Thus, computer hacking, and other computer related mischief undertaken or caused by people with either benign or malicious intent is of grave concern to businesses, particularly those that rely on expansive computer networks. Indeed, a breach of an organization's network, including even a single computer on that network, may result in direct and indirect financial loss, the latter including loss associated with costs such as legal fees, and fines.
The consistent demand for computer and other network services has increased the need for better network security tools. A variety of techniques have been deployed to shield networks from hacking and other intrusions. Those protective techniques may be categorized as either risk avoidance systems or risk management systems.
Risk avoidance techniques involve introducing a barrier to prevent inappropriate entry into a network. Such systems place reliance on keeping intruders out of the network entirely, rather than monitoring inappropriate network traffic after logging in. Risk avoidance systems include dedicated network firewalls and mandatory encryption over the network.
Risk management approaches, in contrast, adopt the philosophy that a network can not keep everyone out, and so rely upon detection of intrusive activity after logging in. A risk management approach uses auditing products that employ so-called sniffer technology to monitor network traffic. Data streams collected by such products look for specific types of network traffic by, for example, detecting electronic mail uploads by monitoring port 25 for simple mail transfer protocol (SMTP) events. However, most networks carry a large amount of traffic and simple sniffer type tools do not help sift through the volume. Other drawbacks exist.
The conventional auditing and analysis systems also fail because they require training personnel to use the numerous analysis tools needed to investigate network communications having many different protocols. This training is expensive. In addition, network analysis continues to become increasingly difficult due to the large number of new applications and protocols being introduced every year. Specific analytical tools must be developed for each collection system making it difficult to cross-correlate events and perform analysis.
Further, security practitioners often have to perform long and tedious tasks in order to complete an investigation indicating a possible security breach. These investigations involve a large number of repeatable manual steps that are laborious to perform.
There is accordingly a need to provide more comprehensive methods and systems that can provide users such as security practitioners and computer network managers the ability to have confidence that security risks are being effectively detected and analyzed.